Decentralized Stochastic Planning with Anonymity in Interactions

In this paper, we solve cooperative decentralized stochastic planning problems, where the interactions between agents (specified using transition and reward functions) are dependent on the number of agents (and not on the identity of the individual agents) involved in the interaction. A collision of robots in a narrow corridor, defender teams coordinating patrol activities to secure a target, etc. are examples of such anonymous interactions. Formally, we consider problems that are a subset of the well known Decentralized MDP (DEC-MDP) model, where the anonymity in interactions is specified within the joint reward and transition functions. In this paper, not only do we introduce a general model model called D-SPAIT to capture anonymity in interactions, but also provide optimization based optimal and local-optimal solutions for generalizable sub-categories of D-SPAIT.Singapore. National Research Foundation (Singapore-MIT Alliance for Research and Technology Center. Future Urban Mobility Program

Solving Uncertain MDPs with Objectives that are Separable over Instantiations of Model Uncertainty

National Research Foundation (NRF) Singapore under SMART and Future Mobilit

Dynamic repositioning to reduce lost demand in Bike Sharing Systems

National Research Foundation (NRF) Singapore under SMART Center for Future Mobilit

Dynamic Redeployment to Counter Congestion or Starvation in Vehicle Sharing Systems

Extensive usage of private vehicles has led to increased traffic congestion, carbon emissions, and usage of non-renewable resources. These concerns have led to the wide adoption of vehicle sharing (ex: bike sharing, car sharing) systems in many cities of the world. In vehicle-sharing systems, base stations (ex: docking stations for bikes) are strategically placed throughout a city and each of the base stations contain a pre-determined num-ber of vehicles at the beginning of each day. Due to the stochastic and individualistic movement of customers, there is typically either congestion (more than required) or starvation (fewer than required) of vehicles at cer-tain base stations. As demonstrated in our experimental results, this happens often and can cause a significant loss in demand. We propose to dynamically redeploy idle vehicles using carriers so as to minimize lost de-mand or alternatively maximize revenue for the vehicle sharing company. To that end, we contribute an opti-mization formulation to jointly address the redeploy-ment (of vehicles) and routing (of carriers) problems and provide two approaches that rely on decomposabil-ity and abstraction of problem domains to reduce the computation time significantly. Finally, we demonstrate the utility of our approaches on two real world data sets of bike-sharing companies.

Sampling based approaches for minimizing regret in uncertain Markov Decision Problems (MDPs)

National Research Foundation (NRF) Singapore under Singapore-MIT Alliance for Research and Technology (SMART) Center for Future Mobilit

Regret based Robust Solutions for Uncertain Markov Decision Processes

In this paper, we seek robust policies for uncertain Markov Decision Processes (MDPs). Most robust optimization approaches for these problems have focussed on the computation of {\em maximin} policies which maximize the value corresponding to the worst realization of the uncertainty. Recent work has proposed {\em minimax} regret as a suitable alternative to the {\em maximin} objective for robust optimization. However, existing algorithms for handling {\em minimax} regret are restricted to models with uncertainty over rewards only. We provide algorithms that employ sampling to improve across multiple dimensions: (a) Handle uncertainties over both transition and reward models; (b) Dependence of model uncertainties across state, action pairs and decision epochs; (c) Scalability and quality bounds. Finally, to demonstrate the empirical effectiveness of our sampling approaches, we provide comparisons against benchmark algorithms on two domains from literature. We also provide a Sample Average Approximation (SAA) analysis to compute a posteriori error bounds.Singapore. National Research Foundation (Singapore-MIT Alliance for Research and Technology Center. Future Urban Mobility Program)United States. Office of Naval Research (Grant N00014-12-1-0999

Models and Algorithms for Stochastic and Robust Vehicle Routing with Deadlines

We consider the vehicle routing problem with deadlines under travel time uncertainty in the contexts of stochastic and robust optimization. The problem is defined on a directed graph where a fleet of vehicles is required to visit a given set of nodes and deadlines are imposed at a subset of nodes. In the stochastic vehicle routing problem with deadlines (SVRP-D), the probability distribution of the travel times is assumed to be known and the problem is solved to minimize the sum of probability of deadline violations. In the robust vehicle routing problem with deadlines (RVRP-D), however, the exact probability distribution is unknown but it belongs to a certain family of distributions. The objective of the problem is to optimize a performance measure, called lateness index, which represents the risk of violating the deadlines. Although novel mathematical frameworks have been proposed to solve these problems, the size of the problem that those approaches can handle is relatively small. Our focus in this paper is the computational aspects of the two solution schemes. We introduce formulations that can be applied for the problems with multiple capacitated vehicles and discuss the extensions to the cases of incorporating service times and soft time windows. Furthermore, we develop an algorithm based on a branch-and-cut framework to solve the problems. The experiments show that these approaches provide substantial improvements in computational efficiency compared to the approaches in the literature. Finally, we provide a computational comparison to evaluate the solution quality of the SVRP-D and the RVRP-D. The results show that the RVRP-D produces solutions that are very competitive to those obtained by the SVRP-D with a large number of scenarios, whereas much less sensitive to the distributional uncertainty

Stochastic Dual Dynamic Programming for Multiechelon Lot Sizing with Component Substitution

International audienceThis work investigates lot sizing with component substitution under demand uncertainty. The integration of component substitution with lot sizing in an uncertain demand context is important because the consolidation of the demand for components naturally allows risk pooling and reduces operating costs. The considered problem is relevant not only in a production context but also in the context of distribution planning. We propose a stochastic programming formulation for the static-dynamic type of uncertainty, where the setup decisions are frozen, but the production and consumption quantities are decided dynamically. To tackle the scalability issues commonly encountered in multistage stochastic optimization, this paper investigates the use of stochastic dual dynamic programming (SDDP). In addition, we consider various improvements of SDDP, including the use of strong cuts, the fast generation of cuts by solving the linear relaxation of the problem, and retaining the average demand scenarios. Finally, we propose two heuristics, namely, a hybrid of progressive hedging with SDDP and a heuristic version of SDDP. Computational experiments conducted on well-known instances from the literature show that the heuristic version of SDDP outperforms other methods. The proposed method can plan with up to 10 decision stages and 20 scenarios per stage, which results in 20 10 scenario paths in total. Moreover, as the heuristic version of SDDP can replan to account for new information in less than a second, it is convenient in a dynamic context